Video special effect systems for spatially transforming two-dimensional real-time or "live" television video images are well-known. These systems are capable of generating two-dimensional video images which can be displayed and transformed in a variety of ways including rotation, translation and perspective manipulation.
A typical television special effects processor operates by first digitizing a normal analog television signal to generate a plurality of digital pixels which represent the digitized picture. The pixels for each television frame are then stored as "source" pixels in a field memory. The source pixels stored in the memory are sequentially used to generate output pixels in each scan line in the output picture. More specifically, a control circuit which has been programmed with the desired mathematical image transformation equations repetitively fetches source pixels from the memory. The source pixels are fetched from memory locations which will generate the next pixel on the output scan line. Using the image transformation equations, a transformation circuit computes the pixel value of the next pixel in order to generate an output transformed video signal. With specialized hardware, the entire process of storing and computing the transformed video information can be repeated at a rate sufficient so that the output video image changes at a television processing rate and produces "live" transformed television images. An example of such a system is shown in U.S. Pat. No. 4,631,750 entitled "Method and System for Spatially Transforming Images" issued on Dec. 23, 1986.
Special effects are normally displayed in sequences that span a series of video frames. In each step in the sequence the video image is transformed from one video field to the next video field in order to generate an overall effect which lasts a predetermined period of time. For example, the image may move, or translate, across the display screen, change size and rotate over a period of time. Such a sequence may involve thousands of slight individual transformations from field to field. Consequently, in order to display a special effects sequence, it may be necessary to carefully specify a long series of sequential transformations which will be carried out over a number of video fields. If the number fields spanned by the transformation is large, the manual specification of each individual transformation is tedious and time-consuming.
Therefore, conventional video special effects systems utilize a method called "keyframing" in order to allow a user to quickly compose effects. A keyframing system has two modes: a "program" mode in which the special effect sequence is defined and set up and a "run" mode in which the pre-defined effect sequence is shown on a live display.
In the program mode, in order to define a special effects sequence, a user defines one or more keyframes. Each keyframe is a set of parameter values which define the video image at a particular point in time during the effect sequence. Each effect sequence must have at least two keyframes: one at the start of the effect sequence and one at the end of the effect sequence. Other keyframes may also be defined between the starting keyframe and the ending keyframe at selected points in the sequence, for example, when the direction of movement changes or one effect changes to another. In addition, in the program mode, the user specifies the time duration between each keyframe in the pre-defined sequence. This time duration may be only one field or may extend across many fields.
After the keyframes have been defined, the special effects system software utilizes mathematical interpolation algorithms to generate transformation specifications for each unspecified video field which occurs in the effects sequence between keyframes. Thus, to create a special effects sequence, the user need only specify only a few keyframes at "key" points in the sequence.
Finally, in the run mode, the user and system specified sequence of transformations is run in a continuous process with live video displayed. After the user is satisfied with the effect, either the keyframe specifications or the entire sequence of transformations is stored for later playback.
For example, a simple effects sequence and the keyframes which define it are shown in FIGS. 21A-21E and 21-1 to 21-3. In the sequence shown in FIGS. 21A-21E, a reduced-size video image starts in a position at the top right corner of the viewing area (FIG. 21A). The image swings down and to the left side of the screen while increasing in size (FIGS. 21B and 21C). The image then moves to the right and further increases in size (FIG. 21D) so that at the end of the effect the image becomes full-sized and is centered (FIG. 21E). This effect sequence only varies three parameters: X, Y and Z position. The three keyframes which define the sequence are shown as keyframes 21-1, 21-2 and 21-3. All a user needs to do to define this sequence is to specify: 1) at what point in the sequence the keyframes occur; 2) what the X, Y and Z positions of the image are in each keyframe and 3) what kind of interpolation algorithms the system software will use to define the intermediate frames between the three keyframes. The system software then computes the image X, Y and Z positions for each field in the effect sequence.
While the aforementioned two-dimensional keyframing system is easy to setup and is capable of generating a variety of digital special effects at a real-time video rate and of simulating certain three-dimensional effects, it does not have true three-dimensional capability. For example, it is not possible with such a system to rotate a three-dimensional object with real-time video displayed on all surfaces as the object rotates.
There are several video special effects systems which are capable of generating true three-dimensional video special effects which can construct a three-dimensional object and then manipulate the constructed three-dimensional object by rotating it, translating it, etc. In the video special effects system described herein, a flat source surface is divided into a plurality of flat polygonal "patches" and a 3D surface is first constructed by manipulating the patches in three dimensions by rotating, translating and scaling each patch to construct the 3D surface. Then, in order to manipulate the entire constructed surface, the vertices of each patch are transformed in order to establish their location in the final transformed 3D shape. Once the patches have been transformed, real time video is mapped to each transformed patch face by using the target points in each patch to determine the necessary source points.
While the aforementioned keyframe method of specifying the desired effect sequence can be used with such a patch-driven system, additional complexities are introduced due to the number of parameters that must be specified in each keyframe. In a two dimensional system, only a small number of parameters such as X, Y and Z position, translation, scaling and perspective are applicable and need be manually specified in each keyframe. However, with the patch-driven 3D system, the parameters for each patch must be specified if the 3D object is to be constructed as part of the special effects sequence. If the 3D object consists of only a few patches, then only a small number of parameters must be specified in each keyframe, but a 3D object which is comprised of a large number of patches or which is curved requires the specification of a large number of parameters in each keyframe resulting in a time-consuming setup operation.
Accordingly, it is an object of the present invention to provide a method for controlling a patch-driven 3D video special effects system in which 3D objects can be quickly and easily defined.
It is another object of the present invention to provide a method for controlling a patch-driven 3D video special effects system which is compatible with conventional keyframe control.
It is still another object of the present invention to provide a method for controlling a patch-driven 3D video special effects system which allows complex 3D objects to be quickly defined with a small number of parameters for each keyframe.
It is yet another object of the present invention to provide a method for controlling a patch-driven 3D video special effects system which allows a 3D object with a pre-defined shape to be changed to another 3D object with another shape in the course of a special effects sequence.